This disclosure relates to computer animation for creating a 3D model, in particular for creating a 3D model based on 2D images.
Acquiring the 3D geometry of real-world objects is generally known in the art. In computer vision, image-based scene reconstruction techniques are used to create a 3D model of a scene, given a set of 2D images of the scene. Many 3D reconstruction techniques are known in the art. For example, passive techniques that analyze a multitude of images of the scene and are referred to as, by those skilled in the art, multiview stereo or photogrammetry methods. These image-based methods can construct a 3D model relatively simply and cheaply by employing standard imaging hardware like consumer digital cameras. These image-based methods can provide color information of the scene and offer high resolution scanning thanks to the advances in image sensors. Most multi-view stereo methods filter, smooth, or denoise the reconstructed depth maps, and often these steps are integrated into the depth estimation stage and formulated as a (global) optimization problem.
One common approach employed by the multiview stereo methods for constructing a 3D model is to first compute camera poses and then estimate depth maps for all views by finding corresponding pixels between views and triangulating depth. Under that approach, all pixels are then projected into 3D space to obtain a point cloud from which a surface mesh can be extracted using point cloud meshing techniques.
However, one drawback of the approach described above is that it is prone to producing outliers and noise in the depth maps due to matching ambiguities or image imperfections (such as lens distortion, sensor noise, etc.). The resulting point clouds under that approach are often noisy. Typically, the meshes computed from such noisy point clouds either miss many details (when a lot of regularization is applied) or reconstruct wrong geometry that often manifests itself in disturbing blobs. A common remedy to reduce outliers in image-based methods is, similarly to the surface reconstruction, to use strong smoothing or regularization in the depth computation, but this inevitably destroys fine details and is also costly to compute as it typically comes down to solving large global optimization problems.